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■ INTRODUCTION ' * 

background and Objectives 

The jjire^j- . iiducacors 'and noneducar.crs alike 
have shown a.»growing awareness of the lack of — 
and need for — evidence as to whether o.r not inno- 
vative educational practices are indeed be t ter 
than the more traditional approaches. 

In response to this need, the U.S. Offi-ce of. 
Edt^cation in 1969 awarded a contract to the 

, American Institutes toi Research to develop a 
design for a study of the effectiveness of highly 
intensive, innovative educational practices on ^ 

^students in grades i' through 12. The general 
emphasis'of the resulting Project LONGSTEP (the 
Longitudinal Study of Educational Practices) was 
on the identification- of changes in student; 
achievement that occur as a result of intensive 
educational innovation, "intensive innovation" 
meaning the implementation of a new program encom- 
passing a significant proportion of students, 
entailing a major alteration of school procedures, 
and involving a high investment 6f resources. 

Objectives . Specific objectives were to design 
a system to study the characteristics underlying 
i^.ii.'vative, educational approaches; to establish a 
large-scale data base of program characteristics 
and stjdent outcomes for "a select sample of edu- 
cational programs involving intensive and highly 
innpv£?tive education practices; to determine longi- 
tudinally the impact of such innovation upon stu- 
dent P'^rformance and attitu^les; and to attempt to 
idtDtify the dimensions of the components that 
exhibited the greatest impact on student outcomes. ^ 

Methods and Techniques 
> 

Data Collection . The data collection instru- 
ments used in this study- provided information on 
student performance in reading, language, and 
arithmetic on standardized achievement tests; 
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student characteristics; educational practices; 
and teacher characteristics. Student cognitive 
^hievement v^s measured by either of two commeT- ^ 
cially developed iastruments . Student and teacher 
background, characteristics and attitudes were 
assessed by questionnaires developed specifically^ • 
for Project LONGSTEP. . ^ . 

The Educational Experience Analysis Guide was 
developed so that complex e^ycational experiences 
could be descriP ^ and ^,fl.^tif ied with respect to 
specific observabTe'^Siaracteristics rather than 
on thd basis of* variously and ambiguously defined 
local labels. This Guide was^used to document . 
the basic educational attributes of school pro- 
grams and to locate Ihe educational experiences 
of participating students on a continuum ranging 
from traditional to innovative. Information on 
educational experiences was gathered from inter- 
views with principals and teachers, from classroom 
ubservations, and from existing school document^c.- 
tion. This methodology, led to the identification 
of more than 200 different, kinds of educational 
experience groups. 

A classroom observation instrument documented 
characteristics such as physical- environment, 
study arrangements, access to' resources, and use 
of materials. 

Data Source, 

' Programs. Schools located in 13 school dis- 
tricts in California, Florida, Kentucky, 
Minnesota, Pennsylvania, Texas, Utah,' Washington,; 
and Wisconsin were selected and became partici- 
pants during the entire three-year (1970-71, 
1971-72, and 1972-73) implementation phase. 
Some 30,000 student^, 80 schools, and 1,300 
teachers were involved in the project during 
Lhese schot>l years. 

Innovations . Educational innovations included 
team teaching, multimedia emphasis, unique^ school 
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design, use of paraprofessionals, variations in 
scheduling, and teacher-developed materials as 
well as independent study, student selection of 
materials, and a number of other practices typi- 
cally associated with individualized instruction. 

Communities » Ihe communities served by the 
school districts ranged from 2,500 to over 600,000 
in population and varied from rural to urban- 
metropolitan in setting with a notalle diversity 
in socioeconomic level. The reported instruc- 
tional cost per pupil in the participating school 
districts varied from a low of $540 to a high of 
$1,050. The percentage of' minority group students 
ranged from less than 1 percent to over 30 percent 



PROJECT REPORTS* 



A total of five project reports plus two separ- 
ately bound appendices of supporting data were 
produced by Project LONGSTEP. The purpose of this 
overview is .to summarize very oriefly these five 
reports . 
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Volume I: Impact of Educational Innovation on 
Student 'Performance: Project Methods and Findings 
for Three Cohorts - Gary J. Coles, Albert B. 
ChaliApsky , Bruce E, Everett, Marion F. Shaycoft, 
Barbara J. Rodabaugh, Malcolm Danoff (April 1976) 

This report focuses on three groups of stu- 
dents: those who started out as first-graders in 
1970-71, those who started out as fourth-graders 
in 1970-71, and those who started out as sixth- 
graders in 1970-71. Its primary purpose was to 
analyze overall differences in achievement growth 
among educational approaches;- an educational 
growth model in which achievement was related to 
innovative emphasis, number of minutes of instruc- 
tion per day, pretest, socioeconomic status, and 
teaching qualifications; a posttest achievement 
score statistically adjusted for pretest aad 



A limited supply of project reports is currently 
available and eventually all repr^rts will be 
available through the Educational Resources Infor- 
mation Center (ERIC) ^ In addition, a set of gen- 
eralized data files was developed and submitted 
to USOE for possible use bi) other investigators. 
For information on the availability of reports 
and the data files, contact Dr. George W. Mayeske, 
Office of Planning , Budgeting , and Evaluation, 
USOE, 400 Maryland Avenue, Washington, D.C. 20202. 
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socioeconomic differences; groups of students wii 
similar educational experiences who, on the aver- 
age, performed' either much better or much worse 
than was expected from their pretest and socio- 
economic status; and students who demonstrated 
particularly large achievement gains during two 
cortsecutiye school years. 

The major conclusions ot this repi^rt were as 
follows: • 

• No evidence could be foupd that either of the 
major treatment variables — Level of Innova- 
tion or Degree of Individualization — was 
substantially and positively related to post- 
tes^ performance. Further, pteliminary 
analyses showed that, on the average, these 
findings applied equally well to students at 
different socioeconotirtc or pretest levels. 
The expectation that substantial yearly gains 
in student achievement would occur for a 
sample of intensive, innovative educational 
programs was not supported by these findings. 

• Even though overall project findings showed 
that dramatic school effects were not associ?- 

v^ated with intensity of educational innovation 
different educational approaches did produce 
meaningful and important differences in 
. : achievement, especially in the early elemen- 
tary grades. Unquestionably large gains in 
reading, language, and arithmetic skills, 
over and above those expected on the basis 
of pretest and socioeconomic status^^were 
, found to occur in some of the participating 
schools . ■ ^ : 

A separately bound executive summary andL' a . 
volume of appendices accompany the basic volume. 

;' 
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volume I Supplement: Impact of Educational .Jnno- 
vat ion on Student Performance: Overall Findings 
for Reading ^d AritJimetUc - Gary J. Coles, 
Albert B. Chalupsky (September 1976) 

The purpose of the analyses conducted for this 
supplement to Volume I of the Project LONGSTEP 
fioal repcrt was (a) to ascertain if the trends 
observed and discussed in Volume I with respect 
to students ii? grades 1, 4, and 6 during the .J.970- 
71 school, year were represeiitaLive of the trends 
shown by all analysis samples and (b) to compare 
results and determine if other meaningful trends 
across cohorts were present. Tlie overall find- ' 
ings reported here ha^ shown the following: 

• The mean reading and^'arichmetic posttest scores 
for Project LONGSTEP's sample of fairly inno- 
vative scHools were not conspicuously farther 
from national norms than their average pretest 
scores were from their norms. 

• Variations, among analysis samples with respect 
to average reading and arithmetic achievement 
gains did not tend to be as'sociated in any 
highly consistent manner with concomitant dif- 
ferences in the mean Level of Innovation, 
Number of Minutes per Day, and, Teaching Quali- 
fications of the samples. 

• Variation in Level of Innovation was not consls- 
cently or positively related to reading acUieve- 
menc within analysis samples. 

• * Variation in Level of Innovation appeared.to 

be negatively associated with arithmetic 
achievement.', 

• Variation in Teaching Qualifications was not 
highly or consistently related to reading 
jic:iievemenc but was positively associated witii 
small gains in arithmetic achievement. 
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In summary, the primary research hypothesis, 
that substantial gains in achievement test perfor- 
mance are positively associated with innovative 
emphasis, has not been supported in any general 
way by the analysis of Project LONGSTEP's data. 
Thejie results, based on a global analysis of trends, 
across reading and arithmetic analysis samples and 
grades^ Ce^nd to confirm the findings reported in 

Volume I for students in grades 1, 4, and 6. 
» • 

A separately bound volume of supporting appen- 
dices to Volume I -Supplement is being deposited 
in the Educational Resources Information Center 



(ERIC) . 




/ 



S 



11 



-7- 



5 



Volume II: Inno\/ative School Environinents and 
Student Outcomes - Gary J. Coles, Albert: B. 
Chalupsky (Sepvember 1976) 

The specific purpose of this volume was to 
explore, the possibility that growth in student.' 
achievement test performance and positive changes 
in school-related attitudes were highly associated 
with- school environments in which there was, on 
the average, a great deal of emphasis on innova-. 
tion.._ Both student outcome scores and treatment 
data in lahguage arts, mathematics, social 
studies, and science were aggregated to the 
school level so that the more general questipn of 
the relar.i6n between school environments and out- 
comes cOuld be explored. 

The findings of .this study suggest that 

• Important differences among schools with 
repect to the achievement test performance 
and attitudes of their students * existed in^ 
a number of LONGSTEP samples analyzed. 

• Greater average growth in .achievement test 
performance and "positive changes -in attitude 
were not associated with school-level empha- 
sis on innovation -and individualization. 

s Measures of growth in acliievement were ty pi- 
tally not related to our quantity of school- 
ing indices. There was, however, a tendency 
for these indices to be positively, related • 
to student attitudes toward schoolings 

• In genera^, chan^ejs in avei^age student atti- ' 
tudes toward s^hool^ were not s-ignif icantly 
related to average growth in achievemenf.: 
However, the majority of. correlations were 
positive. , . 

In respect to our primary hypothesis, the 
results of this study indicate that innovative 
school environments did not demonstrate a sub- 
stantially positive impact on either achievement 
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or student attitudes: These findings essentially 
support the student-level,- f indings reported in 
VolCime 1 and the Volume I * Supplement . The pattern 
of results leads us to- conclude that important 
differences among schools in the LOIJGSTEP sample 
did occur but that such differences were not 
highly associated with innovative school 
environments. 
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Volume II Appendix Report: A Preliminary Studu 
of the Relevance of a Ste^ndardized Test for^ ' 
Measuring Achievement Gains in Innovative Arith- 
metic Programs - Bruce E. Everett (September, 1976T^ 

During the course of Project UONGSTEP, questions^ 
were raised concerning (a) the relevance of the 
standardized^ achievement test utilized in the'pro- 
ject to the stated o^>jectives of the edu'cational 
treatments included in the parti-cipating educa- 
tional programs *and (b) the extent to which jthe 
findings discussed in Volume ^ may. have been 
influenced by the particular instrument chosen\ 
to measure cognitive achievement. To provide at 
least partial answers to these questions, a study 
of the arithmetic items contained in the test and 
their relevance to the educational objectives of' 
rhe arithmetic treatments enjcompas'sed by Project 
LONGSTEP was conducted. 

Specifically, two research questions were, 
addressed: 

• Were the particular skills necessary to 
answer correctly the arithmetic items on 
the Comprehensive Tests of Basic Skills 
(CTBS) actually incorporated into the cur- 
riculum objectives of arithmetic treatments? 

• To the extent that arithmetic treatments 
differed from one another in the relevance 

*of the CTBS test items to their curriculum 
objectives, wh^t effect did this difference 
have on test performance? 

With respect to our^first research question, 
whether or not the arithmetic skills tapped by thp 
CTBS were actually incorporated into curriculum 
objectives of arithmetic treatments, the answer is 
clearly "Yes" for the third- and sixth-grade sam- 
ples selected from Project LONGSTEP.' The CTBS, 
Forms Q and R, does. appear to be a valid measure 
of the degree to which student performance in 
arithmetic matches the arithmetic objectives of 
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these particular treatments. Although we do not 
•know iiow widely these treatments differ among 
themselves in terras^of objectives that do not 
appear on the CTBS, there are very few grade 
level arithmetic skills which are not present in 
some way on the CTBS. What disagreement there is 
about the relevance of CTBS items among the 
teachers polled is largely confined to the more 
complex, difficult objectives, and these objec- 
tives are tapped by a much smaller number of items 
than are thbse relating to basic arithmetic 
skills. • 

In ^response to the second research, question, to 
what extent variations in the relevance of the 
CTBS to curriculum objectives affect test perfor- 
mance, the fact that there is so little variance 
among . treatments in the relevance of the CTBS 
probably accounts for the finding that the impact 
of domain relevance is so slight. Also, the pre- 
test scores for the two samples show themselves 
to be much more important predictors of posttest 
performance than does domain relevance. The 
importance of current exposure to these arith- 
metic skills thus-^pears to be largely over- 
shadowed b;y prior experience with those skills. 



P roject LOi^JSTEP Memorandum Report: Parental 
Educational Expectations and Their Impact on 
Student Outcomes' - Albert B. Chalupsky , Gary 
J. Joies (September 19 J6) 

I 

Thif> report' summarizes the results of an 
exploratory study of the' relationship 'between par- 
ents* educational expectations for their children, 
the children's perceptions of these expectations, 
and student outcomes. Of particular interest 
were the congruence between parental 'expectations 
and the children's perceptions of these expecta- 
tions and the impact of this congruence on student 
achievement and attitudes toward school. 

The data for the present study ^,amo from an 
earlier Investigation designed to determine the 
dependability of the Project LONGSTEP question- 
naire retiponses. Stud^jnts represented a 2% 
stratified -random sample (by gr^de within each 
school) of the students participating ia Project 
LONGSTEP during the 1971-72 school year. 

The results of the present study suggest that 

• Parental expectations (as perc^jved by Che 
children) concerning how far ia schocl they 
want their children to go and how goo: a stu- 
dent they want their children to be were both 
positively related to .children's general*'atti-,- 
tudes toward school but not related to children's 
achievement test performance during the subse- 
quent year. 

• Parental expectations (as reported by parents) 
concerning how far in school they want their 
children to go showed a positive relationship 
to children's attitudes toward school, ;:egard- 
lt2ss of how accurately children perceive these 
expectations. 

• Parental educational expectations concerning 
how good a student they want their child to 
be were positively related to children's 




attitudes toward school in those cases where 
parental expectations and children's percep- 
tions were in close agreement. .Where there 
was very low agreement between parental expec- 
r.ations for how good a student they want their 
cl'iiluitiu Lu be and children * s^ perceptions of 
these (expectations, there was. a negative rela- 
tionship between the parents' expectau.^ons and 
their children's school-related attitudes. 
With moderate agreement between p irentaj. 
expectations and student perce^ r ; r..s, stuient 
attitudes toward school were not ' r^>,\ated iti 
parentaj. expectations. 

• Children appear to be more accurate in estima- 
ting how far in school their parents i«jant them 
to go than in estimating how. good a student 
their parents want them to be . 

• From a practical standpoint, ijarents with high 
expectations concerning how good a student they 
want their children to be wduld be well ar'^v'sed 
to make a special effort to communicate tlcoS': 
expectations to their children. From a ^ 
research standpoint, the degree of congruenr.eV 
between parental expectations and children's \ 
perceptions of how good, a student their parents, 
want them to be may be a worthwhile variable to 
consider for future studies of school-related 
attitudes. 
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